Evaluation of Algorithms Using Games: The Case of Music Tagging
نویسندگان
چکیده
Search by keyword is an extremely popular method for retrieving music. To support this, novel algorithms that automatically tag music are being developed. The conventional way to evaluate audio tagging algorithms is to compute measures of agreement between the output and the ground truth set. In this work, we introduce a new method for evaluating audio tagging algorithms on a large scale by collecting set-level judgments from players of a human computation game called TagATune. We present the design and preliminary results of an experiment comparing five algorithms using this new evaluation metric, and contrast the results with those obtained by applying several conventional agreement-based evaluation metrics.
منابع مشابه
Evaluation erosivity factor of rain using Geostatistical algorithms (case study: Ilam province, Iran)
Today, soil erosion is considered as one of main subjects of watershed management at national and global levels. Assessment of decreasing risk of soil and it spatial distribution are one of key factors for successful evaluation of soil erosion. The objectives of research were calculation and modeling of rain erosivity factor in Ilam province using data of number of 28 pluviometry stations durin...
متن کاملEvaluation of a Music Intervention on Well-being and Behavior of People with Dementia Using Dementia Care Mapping
Introduction: Dementia Care Mapping (DCM) was originally developed as an observation tool to examine person-centered care in long-term care facilities and to evaluate the quality of life and well-being of people with dementia (PwD). However, the effects of a music intervention using this tool have not been investigated so far. This leads to the following research question: How does a music inte...
متن کاملبرچسبگذاری ادات سخن زبان فارسی با استفاده از مدل شبکۀ فازی
Part of speech tagging (POS tagging) is an ongoing research in natural language processing (NLP) applications. The process of classifying words into their parts of speech and labeling them accordingly is known as part-of-speech tagging, POS-tagging, or simply tagging. Parts of speech are also known as word classes or lexical categories. The purpose of POS tagging is determining the grammatical ...
متن کاملLearning to Tag using Noisy Labels
In order to organize and retrieve the ever growing collection of multimedia objects on the Web, many algorithms have been developed to automatically tag images, music and videos. One source of labeled data for training these algorithms are tags collected from the Web, via collaborative tagging websites (e.g., Flickr, Last.FM and YouTube) or crowdsourcing applications (e.g., human computation ga...
متن کاملکاربرد فعالیتهای موسیقایی در مداخلههای کاردرمانی کودکان با اختلال طیف اتیسم
Background: Music is one of the common therapeutic interventions using along with usual therapy in autism spectrum disorder (ASD). This is one of the complementary and alternative medicine coincides with children and has positive effects on them. Therefore, this article aims to indicate some evidence about music application in autism and points the importance of using music in ASD because of th...
متن کامل